DOCUMENT RESUME 



ED 



253^577 



AUTHOR 
TITLE 



INSTITUTION 

PUB DATE 
NOTE 

PUB TYPE 

EDRS PI^ICE 
DESCRIPTORS 



IDENTIFIERS 



ABSTRACT 



* ' TM 85Q lp23 

Brown, Roger; Duffy, John 

Report of the Chapter 1 Sustained Effpcts Study. 
Education Consolidation and* Improvenient Act - Chapter 

^ • 

Cofumbius Public Schools, OH. Dept. of Evaluation 

Services. 

Mar 84 

20p. ' V 

Reports - Evaluative/Feasibility (142> 

MFOl/PCOl Plus Postage. 

Achievement Ga||^hs; *Compensatory Education; • ( 
Elementary Education; ^Lonqitudlnal Studies; Pretests 
Bosttests; Program Evaluation; ^Reading Achievement; 
Reading Tests; Vacations 

*Columbu9 Public Schools OH; ^Education Consolidation 
Improvement Act Chapter 1; Sustained Effects /Study 



school districts receiving 
a study of the sustained 
meet this requirement, the 



Once every three ye^ts. 
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Compensatory Language Experiences and Reading (CLEAR) program 
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«* ' Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 

************************** 



I 



• m 



Bduoatlon Conaollditlon and Inprovenent A^t - Chapter 1 



REPORT OR THE CHAPTER 1 
SUSTAINED EFFECTS STUDY 

Report Date: March, 1984 




Prejpared by: 
Roger Brown^. Profeaslonal Speolallot 
' and 

John Duffy, Profeaslonal Aaalatant 

Under th« Supervision of; 
Richard Ai Aaorase, PheD* 



^ U8 OIPANTMCNTOrCDUCATION 

Naiionai iNsniini of iducatio^j 

I OiJi A!ioN»AlUU SUUIU. J S IIMI Of^MA TlO^ 
M^IIM tlRK » 

MiniM , li.ufij).'. (^,vi« lit'tiM tWinUi (n mijw(»vo 
t|>pt(iilit( lufil i|u,|litY 



V«»Mln.n i>r ptili, V 



■ PERMISSION TO REf^RQDUCE THIS 
MATERIAL HAS BEEN GRANT^ BY 



TO THE EDUCATIONAU RESOURCES 
INFORMATION CENTER (ERIC)." 



ColuBbus (Ohio) Public Schools 
D*Pi^tMnt of Evaluation Sarvloas 
Gary Thomp80|[i, PheD.^ Dlractor 



Kduoatlon Conaolldatlo| and Improvenent Act - Chapter 1 



REPORT OF THE CHAPTER 1 
SUSTAINED EFFECTS STUDY . 

Report Date: Maroh, 1981 



^ ECIA Chapter 1 evaluation guidelines require school districts which are 
eoHving Chapter 1 fynds to conduct a study of^ the sustained effects of 
"hapter 1 prograna* This study muat^ be oondaoted once every three years. The 
istriot .is ribt required to report the results of this study to federal or 
state agencies. The district la Required to keep on file the results of the 
sustained effects study, and to use resjnts in planning. Chapter 1 proff-ams. 

To meet the sustained effects requlreiBent, the Coluabds Public Schools 
employed a fall-spring-fall test design. The purpose of the test design was to 
determine the degree to which the pupils who participated in compensatory 
progriMDs remained at least at the sane level from spring of the treatment year 
until fall of the next school year. 

Pupils from the Compensatory language Experiences and Reading (CLEAR) 
program were Included Ip the study.- The purpose of the CLEAR program was to 
. Improve the language and reading skills of selected underachieving pupils in 
grades kindergarten through eight. 

The present report provides data regarding two questions. First, what 
effect did the 1982- 83 school year have on the achievement levels of pupils in 
the CLEAR program? Second, what was the effect of the time period between May, 
1983 and October, 1983 on the achievement levels of pupils in the CLEAR 
program? ^ ' ^ 

r 

4 

The pupils included in the sustained effects study met- the followli>g five 
criteria. First, the pupils parti oipa ted in the CLEAR^ program in one of the 
Coluipbus Public Sbhools during the 1982-153 sihool year. Mon-public program 
pupils were ^ not included in the study because of the anticipated difficulty of 
locating these pupils the following school year. Second, the pupils were in 
grades one through seven during the 1982-83 school year. Third, each pwpil 
attendfd at least 80^ of the program days between pretest and poattfjst. This 
meant that etoh pupil attended at least '112 of the 110 program days. Fourth, 
pupils were promoted io the next grade at the end of the 1982-83 school year. * 
The sustalnted effects teat was given as part of .the pretest procedures for the 
1983-81 school year. Thus pupils were given the level of the test specified by 
the Chapter 1 evaluation deaign. Pupils who were retained took a lower level 
of the test th^n promotad pupils. A study of the raw sdore* of retained pupils 
indicated that the test level was somewhat low /for these pupils and that a 
celling effect had ooourred,. For this reason, the deoislon wa^ made to drop 
the score? of retained pupils from ^e study. Flft»>, complete protest, 
post test, and sustained effects test d^ta were collected on all of the pupils. 



Table 1 

Nuaber of Pupil « Meeting Various Criteria 

for Inoluslon lo the Sustained Brfeota 
Study Reported by Prograa and Grade Level 



Pupils With Pretest Soore, Post test 
Score, 4 Met Attendanoe Criteria 

-7» 



CLEAR 



162 
857 
565 

322 
596 



Pupils With Sustained 
Efreots Scores' 



Pupils Who Were Promoted 
and Included In the Study 



^ CLEAR 



CLEAR 



73 


73 


670 


609 


32« 


292 


3M2 


311 


266 


.25^ 


522 





312 



268 



2115 



• Table 1 provides a deaorlptlon of the if feot ' of applying these five 
orlterla. The first set of figures shows the nxmber^ of pupils it. each grade 
level in the CLEAH program who were eligible for Inclusion in tne sustained 
effects* study. These pupils met the first three* orlterla. The second set of 
figures shows the number of pupils oji whom sustained affects scores were 
obtained. The third set .<»f figure's shows the number of pupila who met all 
criteria and were Included In the analyses. A comparison of the first two seta 
of figures shows that sustained effects scores were collected on 76 3% of aU 
.•^H^^i* inol-uaion in the study . A oonparlson of the seoona 
and third sets of figures showed that of the pupils having cooplete pretest, 
pbattest and sustained, effects scor,is, 91.9* of the CLEAR pliplla were included 
in the sustained effects jitudy. A comparison of the first and third seta of 
^^"^ ^^"^ eligible for the sustalnfcl effects study 

70. n of the aEAR pupils met the 'five criteria and were included In the study. 

Teat ];natruHi«i^t-g ^- . ' 



The. teat design used the Reading Survey teat ^ of the Metropolitan 

Achievement Tests (MAT; Presoott, Balow, Hogan* A' Farr, 1978) for CLEAR pupil*. 

The norms were established in October, 1977 and April, 1978. Table 2 gives tne 
MAT teat form and level uaed for each grade. 

Procedure ' • . , . 

J 

In the Fall of 1983, the compensatory education Program Involvenient File 
(PIF) for 1982-83 waa used to identify pupils eligible for "inclusion in the 
suatalned effects atudy.. (To be eligible the pupil had to me^t the first Inree 
criteria given In the "Pupils* section of this ' report. ) A printout listing 
neoeaaary pupil and testing Information was produced for each school. 

A ooapenaatory education teacher aasigned to a aohool served aa the 
auatained effeota liaison peraon for t^je building. . The printout and the 
neoeaaary teat materials and Instruotiono were aent, to the llalaon peraon. The 
lialaon peraon waa reaponaible for odlleoting and reporting teat data for 
pupils listed on the printout. Also, the lialaon person collected and reported 
data for any eligible pupil whoae. hame may not have appear^ed on the printout 
for the achool. 



. Tabu 2 

Testing Soha<Jul« for the Rii*ding 
Sua.talnsd Effaota Study f 



. 1982- 83 
Treatment 



1983 . 
Suatmlned Teat 



MatroDQlltan Ar>hif> taiBiint Tmn^ , 



Pre- 



Poat- 



Suatal 




T 



1 

2 

3 
4 

5 
6 



-!f 



2 

5 
6 
T 



Pre,prlmer 
Prlner 
Primary 1 
Prlnar J 2 
Elementary 
Ifitemedlate 



Primer 
Primary 1 
Primary 2 
Elementary ^ 
Int(Brmedlatb' 



■Intermedia 



te 



Primer 
Primary 1 
Primary 2 • 
Elementary 
Elementary 
Intermediate 



•nte^^S X^hT^T? ^•^^ ^Vort.<i by the llalaon pera^na Wbre 

!n.HC!?.«. ?If^^«y»tem. The data were edited, and derlVed ioorea' (afade 

dSS^J^^/ ^rr*"^^^*' ^^^""^^^^ »<'0''«» «o?mal curve ^ equivalent) We 
C^^tr^^nli ^ °' •'^ reaulted In a data file for 

i^utJ.i^.n. ^"-^'^-^ •^'•<^*» »tudy. From theae fllea, normal curve 

D^W nnl??!\""T '*°? ^^^^^^^r dlatrlbujl^ were determinid f or ^e 
?^^r::;re'rta;^^^^ teat,4n,3trlbutlon^of percentile 



RflfliUta 

n„n^i^/°ii'^?i?.i* * ^'^''^ ''^ ^•^••^ reading acorea of the 2,260 

Jn^ii! • ^''^^"'^ ^^^^^'^ 3-5 pertain to the reading 
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Figure 1. A ooaparlbon of the average pretest, posttest, and auatalned effects 
NOS scores for first graders i^^ the Chapter 1 CLEAR program. 

The average pre tea t-^post test HQS gain for first graders In the Chapter 1 
CLBAR progria was 25.2 points. When the sustained «rrfeots testing was done in 
October 1983, the average HCB score dropped 11.0 points. The average MCfc gain 
from pretest to sustained effects test was 14.2 points* Of the 73 first 
gradars in the. study, 19.2$ (14) pupils maintained their NCB level on the 
sustained effects test** 

On the pretest 15.1$ (11) of the 73 first gra\lers in the ^Chapter 1 CLEAR 
program scored above the 36th percentile. On the posttest 67.1$ (49) scored 
above the 36th percentile. On the sustained effects test the percentage had 
dropped to 50,71 (37M^ho scored above the 36th percentile. 
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Figure 2. A oonparleon of the arerage preteat, poatteat, and auatalned effeota 
NCE aooree for aecond gradera In tbe Chapter 1 cysAF progra*. 

nf*?** •v«»*»8« pretaat-postteat NCR gain for aaoond gradera In the Chapter 1 
CLEW? prograa waa 9.5. When the auatalnad effeota taatlng waa done in October. 
19«, the average N(a aoora dropped *.7 polnta. The areraga NOt gain froa 
?^'^^^oo2^""•^^°•'* •ffeota waa 4,8. Of the 609 second gradera in the study, 
36.1^ (220) pupils maintained their HCB lerel on the sustained effeota test. 

On the preteat 20.5f 025) of the 609 seootid gradera in the Chapter 1 CLEAR 
prograa aoored above the 36th percentile. On the poatteat 49.61 (302) puplla . 
acored above the 36th pares ntllt. On the auatalned effeota test the percentage 
had dropped to 36.91^ (225) who scored above the 36th percentile. . 
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Figure 3, A/oomparlaon of tbe ayarum prataat, posttast, andTistalnad effects 
NCE scores for third gradi^s In th^ Chapter 1 CLEAB prograa. ' 



•wage pre test-post test NCB gain for third graders in the Chaptar 1 
CLBAR progra. was 1^2. When the sustained effects testing ^as done In October 
I9tg, the average N(X score dropped. 4. 2 points. The average NGS gain from 
protest to sustained effects test was 6.0. Of the '292 third gradirs in the 
a^udy, 36. 3^ (106) pupils ■ilntalnad their MCB level on the sustained effects 

W e 

On the pretest, 14.4* (42) third graders in the Chapter 1 CLBAH prbgraai 
?fi^h*i.I!^I!4?*'* 36th^pm.centlle. On the poattest 38.4$ (112) scored Jbove the 
IS i^** tf facta test the percentage had dropped to 

24.3% (71) who scorefl above the 36th percentile., . . 
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Figure 4. a ooMparlson of ths average pretest, posttest, and sustained effects 
NCE scores for fourth graders In tl»e Chapter 1 CLEAR prograa. 



m-*?*-™'**^ pre test-post teat MCB gain for fourth graders In the Chapter 1 
CLEAR prograa was 2.8, When the sustained effects testing was done In October . 
1983, th^ average MCB ^oor< dropped 4.4 points. The average NGB scor4 on the 
sustained effects test sEdired^ decrease of 1.6 MCE pplnts fro* the pretest. 
Of the 314 fourth graders In the stjudy,, 37.9$ (1.19) pupils Balntalned their NCK 

level on the sustained effects test. ' 

<t- - ■ \ ' ■• . . 

0n the pretest 16.6JJ (52) of the 314 fourth graders In the Chapter 1 CI^AR 
prograa scored abova the 36th peroentUe. On the posttest 23.65 (T4)'soored 
above the 36jh percentile. On the sustalMd effects test the peroe^itage had 
dropped to 16. 95^ (53) who soorad above the 36th percentile. 
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Figure 5. A ooaparison 6f the iver«ft nreteat, poatteat, and auatalnad affaota 
NCE aooraa foi* fifth gptdara'in the Chapter 1 CLKIR prograa. 

/n».?* pre teat^-poat teat NOS gain for fifth gradera in the Chapter 1 

CLBAR prolan .vaa 7.8, When the auatainad ef facta teatlng vaa dona in October 
1983, the average HOI aoore dropped 4.8 pointa. The average HOS gain froa 
preteat to auatainad effeota teat waa 3.0, Of the 259 fifth gradera in the 
atudy, 36.3$ 194) pupila aaidtainad their^lKX level on the auatainad effecta 
teat. 

On the preteat 3.1$ (8) of the 259 fifth gradera in the Chapter 1 CLEAR 
progran aoored above the 36th percentile. On-tWa poatteat 11.6$ (30) acored 
above the 36th percentile. On the auatainad eff»6ta teat the percentage had 
dropped to 9.3$ (24) who acored abdve the 36th peroentUe. 
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Figure 6. A comparison of tha avarAge pratpat, poattaat, and auatalnad af facta 
MCE aoora'a for sixth gradara In tha Chaptar 1 CLBAR program. 



Tha avarag^ prataat-'poattast NCB gain for alxth gradars In tha Chaptar 1 
CLEAR program waa 8.9. Whan tha auatalnad af facts taatlng was donf In October 
1983, the avaraga N(X had remained tha same. Tha average MOB g^ln from pretest 
^? ^J'^IlS?"* •fftots test was 8.9. .Of the W alxth gradara In the study, 
51.0% (242) pupils malntalnad their MCB level on the austilnad aff^cta test. 

On the pretest 9.51 (45X of tha 4?^ sixth gradefs In the Chapter 1 CLBAR 
program scored above the 36th peroayitlle. On the posttest 19.81 (94) scored 
above tha 36th percentile. On the auatalnad eff^ota test the percent had 
Increased to 24.1Jt (114) of tha students. 
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Figure 7. A ooapariaon of tha avaraga pretaat, poattaat, and auatalnad affaota 
* NOE aooraa for aavanth gradara In tha Chapter 1 CLEAR prograa. 



The average pre teat-poat teat HCB gain for the aavanth gradera In the 
Chapter 1 OUBAR program waa 3.2 polnta. When the austalned effeota waa done in 
October, 19'83» the average MCB acore dropped .7 polnta. The average N(? gain 
froB pretest to sustained effects test waa 2.5 points of the 245 seventh 
graders In the atudy, 46. Of (113) atudenta Mlntaiited their NC8 level In the 
sustained effecta teat. 

• ' 

On the pretest 19*2 (47) of the 245 seventh gradera in the Chapter 1 CLEAR 
prograa scored above the 36th peroentUe. On the posttest, 29.8 (75) scored 
Above the 36th percentile. On the sustained effects test the percent had 
dropped to 24.1 (59) who had acored above the 36th percentile. 
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Figure 8. A oomparlson of the average pre teat, poatteat, and auatalned effeota 
NCE acorea for all puplla In the Chapter 1 CLEAR program. 



The average pre tea t-poat teat NCE gain for all atudenta In the^ Chapter 1 
CLEAR program waa 8.1. Whpn the auatalned effeota teatlng Vaa done In October, 
1983, the average NCS aoore .dropped 2.5 polnta. . The average MCB gain from 
preteat to auatalned effeota teat waa 5.6, Of all puplla In the atudy, 41. 4| 
(938) puplla maintained their MCB lev»l on the auataln^ effeota test. 

On the preteat 1.4.6 (331) of the 2,266 Chapter 1 CLEAR program puplla 
aoored above the 36th percentile. On the poatteat 32.4 (734) scored above the 

5*I!S?"*^^^** suatalnad effeota teat the percentage had dropped to 

26.4 (598). 
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Table 3 



NCE Means and St«n(|ard Deviation 
In Reading for Each of the 
Three Testing Periods 



PXIfldtflflJt * Postteat Sustained' Effeota 



, Grades N X ^ X s,D. ♦ X S.D. 



1 


73 


26.6 


1K li' 


R1 A 


1 0 1 


40 .0 
* 


13.1. 


2 . 


609 


33.0 , 


12.7 


M2.5 


15.2 


37.8 


16.8 
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292 


29.2 ' 


15.1 


39.1} 


13% 2 


. 35 .2 • 


13.9 
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^ 32.1 ' 
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Table l| 

Percent of Pupils. Who Scored 
Above the 36th Percentile 
In Reading 



Grade 



Pre||;eat 



Post teat 



Sustained 
Teat 



1 

2 
3 

5 
6 

1— 

Total 



15.1 

20.5 

^16.6 

V 3.1 
9.5 
1<>.2 
U.6 



67.1 
49.6 
38.1* 
23.6 
11.6, 
19.8 
29.a 



32 



4. 



50.7 
86.9 
211.3 
16.9 
9.3 
a7.ll 
24.1 
26.1* 



Table 5 

Sjimmtry of NCE Average Changes 

and the Percent of Puplla Who 

Maintained Their Aobiev^aent Level 

r 





Number 
Tested 


PPeteBt- 
Poattea^ 


1 


73 


25.2 

4 


2 
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609 


. 9.5 
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292. 


10.2 
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31^1 


2.8 


5- 


259 . 


7.8 


- 6 




8.9 
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215 


3.2 



Na Chingt Maintained 



Poattest-Suatained Preteat-Sulstained t Pbstteat 
KlXftfllifl Jirttfltfl / NCK Laviil 



.U.2 19.2$ 

- ^-7 * i».8 ^ • 36. in 

- ^-2 / 6.0 36.31^ 
-'».'» - 1.6 37'.91t 

- 3.0 36.3$. 

0 8.9 5^.0% 
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^ .7 ^.5 Hb,0% 

All 2,266 8.1 - 2.5 5.6 H^ 



ERIC 



18 



r 



ERJC 



I 



f 1 . n 




Dlaouafllon ' . 

The nidflpiohooX had the greater, per.oentage of puplla who maintained their 
achievement Tevel^froi the prevloua apMng. The alxth and aeventh gradea had 
51.,(3« apd M(>.0% reapeotfully whoae; huatklned ^effeot^ aoore w«a .equal to or 
bottbr than the poatteat aoore. With th^ exception of the flrat grade, the 
percent of elementary' puplla who maintained their aohlement 'level waa 
relatively the aame. Th^ .ranga of aqorea for gradea 2' through 5 waa 36.15^ to 
37.91. The flrat grade had 19.2j( who had maintained their poatteat aoorea!, 
Ho»^ever, the flrat grade had demonatrated the largeat gain from pre teat to 
poattea^, but had Alao ahown the, largeat drop lA NCK polnta from poatteat to 
auatalniid effeota teat. Overall the alxth grade had the leaat amount of q^ange 
from the P^w^eat 1^ t^ agpring to the auatalned *ffeota teat In the fall (aee 
Table 5). * ' 

Comparing the overall NCK data with the 1981-82 atydy, a- amaller decline 
from aprlng to. fall waa rioted^ln the 1983-8^1 data. Specifically, the 19^1-8? 
atudy Indicated a greater "pre-poat" gain and then a" ateeper "poat-auatalned* 
decline than figure 8 of the 1983-84 atudy. ^n Inapeotlon of the 1983-84 data 
for each grade level reveia«cl that gradea 6^ and 7 changed little from 
poattestlng to auatalned effeota testing. Theae two gradea contributed 
significantly to the alower overall rate of decline. 

ffiiBinnrY - 

In aooordance with -the Chapter 1 Federal guidellnea the Columbua Public 
Sohoola completed the auatalned effeota atudy. The reaulta of thlo atudy will 
be uaed in planning Chapter 1 programa. 

-The auatalned effeota deai^n Included a fall-aprlng-fall teatlng aohedule. 
In the Chapter 1 CLBAR program 2,266 puplla were in the evaluation auatalned 
effeota group. Theae puplla took each of three acheduled teata and met the 80 
percent attendanoe criterion of Inatruotlonal daya. 

The reaulta of the auatalned effeota atudy indicate aome decline in all 
grade levela in reading during the ainmer montha, with the "exception of the 
alxth grade. Qradea 2 through 5 had approximately the aame decline in NCB 
points. The flrat grade had achieved the Ijreatest growth at the end of the 
instructional period but also showed the greatest decline. Grades 6 and 7 
ahowed little or no change from the post test. 
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